Method and system for data storage and management

ABSTRACT

Exemplary memory storage system and methods for distributive storage of data. Exemplary embodiments provide methods and systems including a plurality of nodes where each node has memory for storing data. The nodes may be configured to receive data and store the data at the node if the data is intended for the node or pass the data to another node if the data is not intended for the node. The nodes may manage memory and allocation of specific memory addresses locally, while the system of nodes manages memory based on a naming convention to indicate the nodes and not the individual memory addresses within a node.

PRIORITY

This application claims priority to U.S. Provisional Application Number62/798,931, filed Jan. 30, 2019; U.S. Provisional Application Number62/798,940, filed Jan. 30, 2019; U.S. Provisional Application Number62/798,947, filed Jan. 30, 2019; and U.S. Provisional Application Number62/798,964, filed Jan. 30, 2019, each of which is incorporated byreference in its entirety herein.

BACKGROUND

FIG. 1 illustrates an exemplary surveillance system 100 in which aplurality of cameras 102 capture images from a viewing area 104 andtransmit the data from the captured images along data communication path106 to a storage unit 108. The captured images may be captured stills oras a video feed. The captured data is communicated over a cabledconnection to a data cabinet to be captured and recorded. Each of thecameras capture individual data streams that correspond to therespective viewing area 104 and store the respective data streams in thestorage unit 108. The storage unit may be any conventional storagesystem, such as a redundant array of independent disks (RAID) in whichmultiple hard drives are stored together, to provide sufficient storagefor the data streams of the captured images from the plurality ofcameras. If a video feed is needed, then a user goes to the storage unit108 and retrieves the stored information for the corresponding camera ofinterest.

The surveillance system environment is used as an exemplaryrepresentation of conventional applications requiring large data storageand management. Conventional systems require large spaces for thestorage unit 108, such as in a closet that is subject to high energycosts and temperature control. The stored information is alsovulnerable; as the entire system storage is located in a single spacethat can be destroyed, such as, for example, fire.

There are many products on the market for managing data across one ormultiple networks. RAID controllers and servers have been deployed fordecades with many different configuration options. A challenge is cost.Simple off the shelf style RAID systems start above $1,500 and get ashigh as a cluster cost in the hundreds of thousands of dollars for asmall data center. The second biggest challenge is integration of thesesystems into networks and operations. While RAID serves its purpose, itis costly and quite difficult to deploy. The third challenge becomes theGAP for small businesses and lack of integration without spending afortune on RAID.

In order to be cost effective, many local administrators simply deployNetwork Attached Storage (NAS) units, or small RAID systems not reallybuilt for data vaulting. Companies quickly outgrow these setups and endup duplicating costs in replacing the outgrown system. Dropbox andGoogle Drive are exemplary cloud options, but they require a significantamount of bandwidth to permit real time access to shared files. Thesesystems do not provide a local option. Instead, everything must gothrough the host server first, with no exceptions. Many NAS and smallbusiness grade RAID systems have faulty integrations with these cloudsystems. Remote hosted storage and cloud solutions also face thechallenge of data protection.

Costly wired systems and centralized networks create the necessity forexpensive installs and upgrades. These limitations also lock customersinto a 5-10 year system that will likely be outdated in a much shortertime frame with the given market speed of surveillance innovation anddata storage.

SUMMARY

The present embodiments provide exemplary methods and systems for datamanagement. Although exemplary embodiments are described herein in termsof surveillance management, other applications are also foreseen.Although exemplary embodiments described herein include different systemconfigurations, components, and protocols, exemplary embodiments are notso limited. For example, exemplary applications may use any portion,hardware, software, component, algorithm, or part described herein inany combination. Exemplary embodiments may use the distributive storageand/or redundant wireless communication in this or other applications.

Exemplary embodiments include a plurality of a data generating devicesand a plurality of nodes. The nodes may be integrated with the datagenerating devices or communicatively coupled thereto. The device/nodecombination may include a processor, memory, and one or morecommunication ports. Each node may include software, hardware, or acombination thereof that is configured to store data received by one ormore of the devices, communicate to one or more of the other devicesand/or nodes, pass information from one or more of the devices/nodes toone or more other of the devices/nodes.

Exemplary embodiments may include a data generating device andassociated data storage systems and methods. Exemplary data storageinfrastructure system and methods may permit systems to avoid orminimize using remote storage, such as in a data storage closet or on acloud system. Such reduction in storage requirements or local storage ina singular location may reduce costs in storage, heating/cooling,network management, cabling, etc.

DESCRIPTION OF THE FIGURES

FIG. 1 illustrates an exemplary security system configuration.

FIG. 2 illustrates an exemplary system configuration for a securitysystem according to embodiments described herein.

FIG. 3 illustrates an exemplary system architecture to representexemplary data flows according to embodiments described herein.

FIG. 4 illustrates an exemplary system architecture to representexemplary data flows according to embodiments described herein.

FIG. 5 illustrates an exemplary system architecture to representexemplary data flows according to embodiments described herein.

FIG. 6 illustrates an exemplary system architecture to representcommunication paths and methods according to embodiments describedherein.

FIG. 7 illustrates an exemplary system application to represent thedetection of a tag in proximity of one or more nodes.

DETAILED DESCRIPTION

The following detailed description illustrates by way of example, not byway of limitation, the principles of the invention. This descriptionwill clearly enable one skilled in the art to make and use theinvention, and describes several embodiments, adaptations, variations,alternatives and uses of the invention, including what is presentlybelieved to be the best mode of carrying out the invention. It should beunderstood that the drawings are diagrammatic and schematicrepresentations of exemplary embodiments of the invention, and are notlimiting of the present invention nor are they necessarily drawn toscale.

Exemplary embodiments are directed at a surveillance system. The systemmay include a plurality of data generating devices, such as cameras, orother detector. The system may include a plurality of nodes, in whicheach data generating device corresponds to one of the plurality ofnodes. Other combination of nodes to data generating devices may also beused. The nodes may include memory to store the data received by thedata generating device. The nodes may communicate to each other andother portions of a back up system and/or network through one or moreprotocols. In an exemplary embodiment, the nodes communicationwirelessly.

Although embodiments of the invention may be described and illustratedherein in terms of a camera surveillance system, it should be understoodthat embodiments of this invention are not so limited, but areadditionally applicable to other combination of data sources.Furthermore, although embodiments of the invention may be described andillustrated herein in terms of a system having distributed storageand/or redundant communication protocols, it should be understood thatembodiments of the invention may use any combination of features. Forexample, embodiments of the distributed storage system may be used inother applications outside of the surveillance system, and/or with orwithout the redundant communication. Similarly, the redundantcommunication methods described herein may be used in other applicationsoutside of the surveillance system, and/or with or without thedistributed storage.

Surveillance System / Camera

FIG. 2 illustrates exemplary system configuration according toembodiments of the system described herein. The system configuration 200includes a plurality of data generating sources 202 coupled to aplurality of nodes 208.

As shown, the system may include a plurality of data generating sources202 comprising cameras and a plurality of nodes 208. The system may havea node 208 corresponding to each device 202. Each node/camera pair maybe communicatively coupled either directly or through acommunication/network device. As shown, the camera, node, andcommunication device may be integrated or separated in any combination.For example, the data generating device 202 may be separate from andcommunicatively coupled to a node 208 with an integrated communicationdevice. The system may also use cameras 210A, separate from the node210B, and separate from a communication device 210C, in which each iscommunicatively coupled either directly or indirectly to the othercomponents. The camera, communication device, and node may be a singleintegrated unit 212. The system components may be coupled together orcommunicatively coupled, such as through Ethernet, USB cable, WiFi, orother wired or wireless connection. However, each node, camera,communication device may be integrated and/or recombined in any way andremain within the present disclosure. Accordingly, the camera,communication device, and/or node may be integrated into a single houseor may be contained in any of one or more separate housings.

Exemplary embodiments include a plurality of data generating devices202, such as cameras configured to receive images of an area 204 andgenerate a data stream from the captured images. Cameras are illustratedfor sake of illustration only. Any data generating device may be used.The data generating device may include sensor data in addition oralternatively to image data. Exemplary embodiments may include imageand/or audio data in different ranges, such as to for night vision orheat vision. In an exemplary embodiment, video, still image, camera,audio, or other surveillance data is captured and transmitted throughthe system according to embodiments described herein.

In an exemplary embodiment, the camera may be configured to receive andcapture images on a continuous or non-continuous basis. In an exemplaryembodiment, each camera 202 and/or node 208 may be configured to detectmotion and/or proximity of an object within its viewing area 204. Oncedetected, a camera may be configured to capture images received by thecamera. Such a non-continuous recording may reduce data storage and/ortransmission overhead. If the system is configured for non-continuousdata capture, the system may create storage offsets for cameras/nodes inareas that are highly trafficked, generating and capturing more data,than cameras/nodes in less trafficked areas generating and capturingless data. Accordingly, as illustrated in FIG. 2 , some camera/nodes mayreach storage capacity faster than others.

Exemplary embodiments include a plurality of nodes 208, 210B, 212. Theplurality of nodes may include a processor and memory. In an exemplaryembodiment, a camera or other data generating device may create a datastream. The node may receive the data stream as received data and storethe received data within memory. In an exemplary embodiment, one or morenodes correspond to one or more cameras, such that a camera stores to agiven node. Exemplary embodiments may also use a distributive storagemethod according to embodiments described herein. The system maytherefore detect the storage level at one or more nodes and make storagedecisions about a given received data and generate storage instructionsaccording to the detected storage and storage decision.

Exemplary embodiments described herein may be used to createdecentralized Power over Ethernet (PoE) Network nodes and cameras thatmay allow the end user to upgrade their surveillance system giving themthe ability to install new security cameras and IT infrastructure inpreviously impossible or costly areas without the need for a centralizedIDF or Server Room.

Storage

In an exemplary embodiment, the system 200 may be configured to detect astorage level at a node and redirect incoming data from a camera/node toanother camera/node or other location for storage. For example, theproximate node to a camera may receive incoming data, but may have afull memory (or other predefined used and/or available storage amount).The node may redirect the incoming data to another storage locationafter the predefined storage threshold has been met or surpassed. Asillustrated by the purple line on FIG. 1 , the redistribution may be toan offsite storage location, such as a cloud storage location. Otherstorage locations may be used, such as a conventional centralized and/oron-site storage location. As illustrated by the green line on FIG. 1 ,the redistribution may be one or more other nodes/cameras of the systemthat has available storage space. The system may use the redistributionas a redundant storage solution. For example, the system may beconfigured to store data locally at the node, while also sending andstoring a copy remotely on a remote storage device, such as a cloudstorage area.

Distributed Storage

FIGS. 3-5 illustrate exemplary embodiments of a distributed storagesystem in which the data generated form a data generating device isstored on one or more memories at one or more locations.

In an exemplary embodiment, the system may be configured to detect astorage level at a node and redirect incoming data from a camera/node toanother camera/node or other location for storage. For example, a system300 may include a plurality of cameras and a plurality of nodescomprising memory. A first node 302 detects a viewing area 304 andgenerates data 306 that is received by a node 308 positioned proximateto a first camera 302. The memory M of the first node 308 may be full(or other predefined used and/or available storage amount). The firstnode 308 may redirect the incoming data to another storage locationafter the predefined storage threshold has been met or surpassed. In anexemplary embodiment, the redistribution may be to an offsite storagelocation, such as a cloud storage location 316. The redistribution maybe through a network 314, such as the internet. Other storage locationsmay be used, such as a conventional centralized and/or on-site storagelocation 318. In an exemplary embodiment, the redistribution may be oneor more other nodes 312 and/or cameras 310 of the system that hasavailable storage space. The system may use the redistribution as aredundant storage solution. For example, the system may be configured tostore data locally at the node, while also sending and storing a copyremotely on a local and/or a remote storage device, such as a cloudstorage area or conventional RAID system.

Each node may be configured to receive and store data received by othercameras. Exemplary embodiments of a node may include a small form factorcomputer including a processor, memory, and operating system. Exemplaryembodiments of the system 400 and method described herein may receiveone or more data streams 406 from one or more data generating devices402 (such as the camera described herein). The one or more data streamsmay be fragmented and stored across one or more nodes of the system412A, 412B. For example, a data stream D1D2D3D4D5D6D7D8 may be receivedat a first node 408 in which the memory M is full. The node may forwardthe data stream to a second node 412A in which the memory is partiallyfull and a portion of the data stream, D1D2D3 can be stored. The secondnode 412A may forward the remaining data stream D4D5D6D7D8 to a thirdnode 412B with empty memory available to store the remaining datastream. The fragmented data stream from the original data generatingdevice may thereafter be stored across a plurality of nodes of thesystem. Exemplary embodiments may include a system and method forfragmenting data streams. Each fragmented data stream may includemetadata for reconstructing the fragment with other fragments toreconstruct the data stream. The system may also be configured to sendredundant or backup data streams to offside storage 406 or other storagelocations 418.

Exemplary systems according to embodiments described herein may includea software controller at each node 502, 512A, 512B of the system 500.The software controller may be configured to generate an aggregated mapof all available space of each node on the system. The softwarecontroller may then determine how to fragment a data stream based onavailable space and provide instructions and associated metadata foreach fragment to identify where the fragmented piece should be directedthrough the system and the node to which it will be stored. For example,as seen in FIG. 5 , the software controller at node 502 receives areceived data stream of D1D2D3D4D5D6D7D8 and has information about thestorage levels at the first node 502 of approximately half full, asecond node 512 of approximately half full, and a third node 512B thatis empty. The software controller of the first node 502 may fragment thedata stream such that more of the data stream is sent and stored at thethird node 512B than at the other two nodes 502 and 512A. Each of thestored data fragments may include metadata (illustrated as an * in FIG.5 ). The metadata of each fragment may be used to identify how toreconstruct the data stream from the data. The metadata may includeinformation of location fragments to create the data stream and theordering of the fragments. For example, the metadata illustrated in FIG.5 may include the location of the first portion of a data stream storedat the second node 512A, the second portion of the data stream stored atthe first node 502, and the third and final portion of the data streamstored at the third node 512B.

Exemplary embodiments described herein may be used to provide anagnostic distributed file system. Exemplary embodiments may use storagespace available at a plurality of different memory locations, eitherwithin a single device or across a plurality of devices. Exemplaryembodiments are configured to use available storage detected on thesystem and sending data for storage on an available memory area withouta specific address of the memory location being used to send the data.

Exemplary embodiments of the system described herein may permit eachnode device, whether it is a Windows, Mac, Linux computer, securitycamera, phone system, other system, or any combination thereof,installed in the network to automatically or manually add storagecapacity to the file storage system. Exemplary embodiments areconfigured to receive information about available memory capacity atdifferent nodes of the system, without specific addressing of theavailable memory. Exemplary embodiments are configured to send data to anode having available storage through the identity of the node and not aspecific memory address.

Exemplary embodiments may use different algorithms for determining howto fragment data strings and/or where to store data fragments. Forexample, as seen in FIG. 3 , data may be received at a node and theavailable memory at the node determined. If sufficient space isavailable, the data may be stored at the node. As illustrated in FIG. 4, if insufficient space is available, the data string may be fragmentedto store the an amount to fill the available space and the remainingfragment passed to the next node. The next node may perform a similarassessment. Information about the storage locations of the fragments maybe relayed back such that the metadata of the fragments may be updatedaccordingly to identify the locations of respective fragments.Alternatively, fragments may be indexed and each memory controller maykeep a master memory location databased of which indexed fragment is ata node. The data stream may also be fragmented upon receipt based onother attributes beyond memory availability. For example, data streamsmay be fragmented to improve data integrity in the event any one node iscompromised or unavailable. An example of segmenting data is illustratedin FIG. 5 in which the receiving node 502 fragments the data stream andthen propagates the fragments to respective other nodes. The fragmentingmay be determined based on the data stream size, communication relaytiming, communication relay bandwidth, the available memory spaces atone or more nodes, a desired maximum data fragment size to improveintegrity in the event of node failure, and combinations thereof.

Exemplary embodiments are configured to save data in file formats inwhich the memory locations used for different files or portions of afile is unknown to at least some of the nodes or all nodes of the systemexcept the node in which the memory resides. In an exemplary embodiment,the system is configured to store memory to specific memory filelocations. Each node may have information about which node correspondsto a file folder or portion of a file folder for storing information.Each node may not know the corresponding specific address locationwithin the node that corresponds to the file folder or portion of thefile folder. The local node in which the memory resides for the specificavailable memory may be managed locally by the operating system of thelocal node. In this way, the overall system may send universalinstructions for managing data locations without specifically requiringan individual memory location or the specific language or protocols ofmemory storage of the individual device providing the available memory.

In an exemplary embodiment, the mapping of specific memory locations ismanaged only locally on the local device providing the memory, but isunknown or unavailable to the rest of the system or other nodes. A datastorage construct of folders is provided herein as exemplary only. Anymemory location or configuration may be used and fall within the scopeof the present invention. The folder structure is used as an example ofa naming convention used to identify memory locations separate from anactual memory location.

Exemplary embodiments of a system may include each node with a memorycontroller communicatively coupled thereto. A memory controller may be aseparate hardware device communicatively coupled thereto to provideaccess to the memory of the node to the memory controller or may havethe memory controller integrated directly into the node throughincorporation of hardware, software, or a combination thereof. Thememory controllers may be configured to store data locally at the memoryavailable from the node or send data to another node for storagethereon. Each memory controller may be configured to know a namingconvention that identifies available memory by the node providing theavailable memory. Each memory controller may be configured to send dataor pass through data to another node for storage based on the namingconvention without knowing the specific memory locations within thereceiving node. Upon receipt by the receiving node or memory controllerat the receiving node, the memory controller and/or local operatingsystem may be configured to locally store the data according to theavailable space. Each local node or memory controller corresponding to alocal node may therefore have access to specific memory locationsavailable to the node.

Exemplary embodiments may include fault tolerant functions withredundant fault controllers and metadata systems that may allow accessto the stored data at all times, no matter the circumstances, or allowaccess under select circumstances.

Exemplary embodiments may incorporate cloud system storages to provideredundant backup and/or add additional storage nodes or capacity intothe network.

Exemplary embodiments may use multiple hardware and softwareconfiguration options that allow the solution to be deployed into anybudget or network design. Exemplary embodiments can store across thecloud, use a storage controller in hardware and software installedlocally, a software controller installed on the devices already on thenetwork, or any combination that works within their network andoperation. Exemplary embodiments of the storage controller anddistributive storage methods described herein may permit a user to takeadvantage of storage already available and present on a system. Forexample, a location may have ten personal computers, such as desktops,on a network, and exemplary embodiments of the distributed storagesystem and methods described herein permit the system to determine ashare size per device and automatically aggregate assigned capacity intothe network growing the storage capacity. This same network maycommunicate with ten security cameras as well, which can also be addedinto the storage array. Such efficient use of storage already availableon a network permits a lower cost solution, while maintaining datasafety, integrity, and availability.

Exemplary embodiments of the storage system and method described hereinis a distributed fragmenting file storage system built upon acombination of Distributed File System (DFS) and New Technology FileSystem (NTFS) data protocols accessible to client operating systemsacross each storage node. In this example, a storage node is defined asa creatable container on the NTFS protocol within each host operatingsystem. This is any file folder created on the node’s host operatingsystem. This system works within Windows, MacOS, and Linux basedoperating systems. The protocols and file folder structure and namingconventions are illustrative only. A storage node is not intended to belimited to any specific protocol of embodiment described herein.

Exemplary embodiment may include storage control software utilizing aJava Applet installed on each client device telling the host device toseek out other applets within the same networked subnet and communicatewith each other for command instructions, passing these instructions tothe host operating system through the protocols within each operatingsystem (OS) and node. This allows exemplary embodiments to create atunnel for data to flow through pre and post marked routing packets inand out of the applet to instruct the operating system to read or writedefined bits of data to a specific folder within the host operatingsystem and allows the host operating system to communicate throughdriver software designed specifically for the hard disk and operatingsystem, allowing the operating system to manage communication for sectordata to the physical disk. Exemplary embodiments of the storage controlsoftware of the applet does not have to communicate to the hard disk inthe same way. Instead, hardware communications happen and may becontained within the operating system of the individual node. Thestorage control software is configured to read and write a file in a bitby bit fashion and appends meta data mapping to each bit passed throughthe applet to each folder within the network of storage nodes (folderscreated on the operating system assigned for communication to theapplet). This meta data is passed across the applet tunnel and may bestored in its entirety on each node for redundancy and file recovery.Each file may be compared via Application Programming Interface (API)against the health of each host system and then balanced across thechosen nodes for each bit with meta data mapping being reported back toevery node in the network through the meta ledger.

Exemplary embodiments may use a modified version of a simple compressionalgorithm on the LV4 open standard to fragment the data and create themeta data mapping instructions.

An exemplary embodiment may look like:

(1) Meta Buffer +--------+ | Line#1 | +---+----+    v {Out#1} -> Meta Ledger 1 (2) Prefix Mode Dependency (Meta Buffer 2)    +----+     | |     v | +--------+-+------+ | Line#1|| Line#2 |+--------+---+----+         v       {Out#2} -> Meta Ledger 1+2 (3)Prefix Prefix     +----+ +----+     | |     v | v |+--------+-+------+-+------+ | Line# 1 | Line#2 | Line#3 |+--------+--------+---+----+              |              v            {Out#3} -> Meta Ledger 1+2+3 (4)       External Meta Write -> Combined Ledgers        +----+ +----+        | | |        v | v | ------+--------+-+------+-+--------+    | .... | Line#X | |Line#X+1 | ------+-------+--------+-----+----+                  v               | {Out#X+1} :=O{(N/B->log->M/B<-N/B)}T                |             Reset (5)                    Prefix                   +-----+                    | |                    v |------+--------+--------+----------+--+-------+    | .... | Line#X | Line#X+1 | Line#X+2 |------+--------+--------+----------+-----+----+                ^|               | v               | {Out#X+2} := O{(N/B->log->M/B<-N/B)}T                |             Reset

Exemplary embodiments of the control storage software may be configuredto write the outputs and inputs of each algorithm running, (2 algorithmsrunning, one for write compression and one for read decompression inreverse order to reconstruct the compressed fragmented mapped datasetsas told by the meta data mapping). The file may be reconstructed upondecompression.

(1)               Reset                    |                    |                   |    Prefix Meta Read                    +-----+                   | |                    v |------+--------+--------+----------+--+-------+    | .... | Line#X | Line#X+1 | Line#X+2 |------+--------+--------+----------+-----+----+                ^|               | v                | {Input#X+2}                | (2)            Internal Meta Read <- Combined Ledgers        +----+ +----+       v | v | ------+--------+-+------+-+--------+    | .... | Line#X | |Line#X+1 | ------+--------+--------+-----+----+                  v                | {Input#X+1}                |             Reset (3)     Prefix Prefix     +----+ +----+     v | v |+--------+-+------+-+-----+ | Line# 1 | Line#2 | Line#3 |+--------+--------+---+----+              |              v            {In#3} <- Meta Ledger 1+2+3 (4)Prefix Mode Dependency Read <- (Meta Buffer 2)     +----+     | |     v+--------+-+------+ | Line#1|| Line#2 | +--------+---+----+         v      {In#2} <- Meta Ledger 1+2 (5) Meta Buffer Read +--------+| Line#1 | +---+----+    v  {In#1} <- Meta Ledger 1      +- - + —<     API<- {Out#1} := O{(N/B->log->M/B<-N/B)}T <- File Reconstruction

Utilizing the input and output compression and decompression schemingresults of the algorithms, the software makes read and write requests tothe host node’s folder through the applet to the operating system, andthe operating system returns the request through the applet back to therequesting node’s operating system compressing or decompressing the filedepending upon if it is a read or write request.

Upon installation of the Java applet into the host operating system,digital container sets may be created within the NTFS processes runningwithin the host operating system allowing for the initial write of themeta ledger to the assigned folder from the operating system. Theoperating system treats this folder as a mounted NTFS disk throughprocedures and protocols built into each host operating system by thecreator of the operating system. Essentially, the operating systemtreats the selected folder no different than a network shared folderthrough SAMBA (which is a standard Windows interoperability suite ofprograms for Linux and Unix) translating to the NTFS protocols allowingfor shared network folders across multiple operating systems. Exemplaryembodiments of the control storage software and data controller runningwithin the Java applet will take the mounted folder in each nodecommunicating from the host operating system and assign packet marks tobuild the meta ledger to assign bit marks to each appropriate node. Thisallows these bits marked to be mapped to the appropriate folder and nodeas the software compares the input and output of the compressionalgorithms running on each applet to choose node storage locations bestsuited for each bit. This also allows the capacity of each node added tothe network to be reported to the software controller running on eachnode, and aggregate available storage capabilities of each node into asingle virtual storage container within the software controller. Shoulda packet be returned to the software controller as undeliverable, thealgorithms are retriggered through a script to check for faults withinthe data marks and redeliver the marked data bit to a different nodethrough the applet. Any returned data bit is considered a faulty bit,and a cron job running every millisecond returns to the node OS that thealgorithm needs to find a new home for that bit. This allows for a nodeto have a varying file capacity to be added into the available storagecapacity in the software controller.

In an example system, Node 1 may have 1 gig of storage, Node 2 may have1000 gig of storage, Node 3 may have 2000 gig of storage, which resultsin total to 3001 gig of available storage to all of the softwarecontrollers running within the applet ring (collection of nodes). Thisability allows for large storage appliances (centralized storagesystems, etc.) to be added as long as they have either a host operatingsystem compatible with the Java Applet. This will allow non-traditionalcomputing devices to be able to add storage capacity to the networks aslong as they are capable of running a compatible OS and Javaenvironment. For example, refrigerators with smart options running onLinux build of Android OS, or access points in the network running aLinux distro may be used to expand available storage on a network.

Exemplary embodiments of the system described herein may permit thedistributed storage similar to RAID, but operates differently than theRAID protocol. RAID uses a software meta mapping controller that isself-contained that needs to communicate directly with the hard drivesto determine the stored sector information of each file within theRAID’s metadata. Exemplary embodiments described herein may use SAMBAprotocols, not hardware protocols, to map the meta data of each file.RAID stripes data to drives, but, again, RAID is contained within theneed for sector mapping data for the file, and it also is writing theentirety of each file to a set of duplicated sectors to multiple drives.This is why RAID with striping requires additional drives specificallyfor a redundant write of the file. In RAiD5, there are main drives andthen slave drives for file redundancy after compression. The compressionalgorithms of the exemplary embodiments described herein are notcompatible with RAID due to the fragmenting of data across multiple hostfolders and not drive sectors themselves.

Transmission

As described previously with respect to FIG. 2 , data generated by thecamera 210A may be sent by the camera and received at the node 210Blocal to or corresponding to the camera. Each node may include memoryfor storing data received by the one or more cameras. Each node may beconfigured to receive and store data received by its immediatelyproximate camera.

However, each node may be configured to transmit data to one or moreother nodes of the system, and/or remotely to other networked locations.Exemplary protocols for transmitting data may include, withoutlimitation, TV White Space (TVWS), Long-Term Evolution (LTE), Ethernet,WiFi, optical, or other wireless or wired methods. Exemplary embodimentsmay incorporate redundant or dual communication links between one ormore nodes. For example, a node may communicate with another node usingLTE and WiFi to improve throughput by providing a redundant transferscheme. Each node (or any combination of nodes) may include errorchecking to validate the incoming data before transmitting and/orstoring the received data. Exemplary embodiments may also includeredundant or different routing schemes such that packets of datafragments from a data stream may be sent along different routes orinclude transmission through one or more different nodes than one ormore other data fragments from the same data stream.

In an exemplary embodiment, different protocols may be used based ondifferent criteria. For example, the speed of transmission, length oftransmission to another node, available bandwidth, security, and anycombination thereof may be considered in selecting which one or moreprotocols may be used to transmit data from one node to another.

Exemplary embodiments of the plurality of nodes described herein mayinclude a cluster of link server node devices configured to deliveruninterrupted high definition (up to 4k) video streams wirelesslythroughout a secured decentralized network. The link offers a fullyflexible, comprehensive surveillance system at a fraction of the cost ofa traditional install. It may allow the end user to store footagelocally, across a local virtual storage platform, and over a cloudstorage service for a triple failsafe storage environment (or anycombination thereof). The link server node may be based on a Power overEthernet (PoE) concept, which allows easy installation, eliminates theneed for costly IDF or Server Rooms. Each node can be accessed andcontrolled independently or as an entire system. The TVWS (or LTE orother communication) umbrella may permit an uninterrupted WiFi streamgiving the user an improved strong signal and reduced system downtime ascompared to conventional systems on the market.

FIG. 6 illustrates an exemplary system set up according to embodimentsdescribed herein. The system 600 includes a plurality of cameras 602that capture images from a view area 604. The plurality of nodes 608 mayinclude a communication module 606 incorporated therein or coupledthereto. Exemplary nodes may include a plurality of communicationinterfaces. One communication interface may be used to send and receivedata to a coupled device. The one communication interface may be a wiredor wireless interface. The node may be configured to integrate one ormore devices into a mesh network without requiring multiple cablingaccess points for each device. Exemplary embodiments may use acommunication module 606 having two (or more) communications interfaces610, 612 for sending and receiving data wirelessly to one or more otherexemplary nodes and/or to a wired connection point. The system may beconfigured to communicate through the two or more communicationsinterfaces using different protocols. As illustrated, the communicationinterface 606 may include a first communication interface 610 and asecond communication interface 612. When the communication interface 606sends data through the communication module 606, it may do so over bothcommunication interfaces, or any combination of the communicationinterfaces. Each node may support multiple Infrastructure SSIDs. Eachradio or communication interface may include an independent radio, suchas 2.4 GHz and 5.8 GHz radios configured in a mesh and/or infrastructureconfiguration to meet local service requirements. Each node may supportPoE plug in 616.

In an exemplary embodiment, the system may be configured to bond 618 thetransmissions through the two or more communications interfaces to sendand receive redundant data for improved transmission time and/oraccuracy. The system may be configured to provide handshaking signalssuch that the transmission of a data set is synchronized or verified fortransmission losses. For example, if a data packet is sent across thetwo or more communications interfaces, the system may be configured tocheck a transmission loss on a receiving end of the communication link.If the system determines the data loss is within an acceptable limit,the system may proceed with the transmission that was first received andnot wait for the transmission through the second communicationinterface. If a transmission loss is detected, then the system may beconfigured to receive and check the transmission through the secondcommunication interface. The combination may therefore be used toimprove time throughput, as the system can use the transmission thatoccurred the fastest, but still benefit from reduced transmission lossesthrough the redundant system. The system may accommodate different loadsacross the different communication protocols. For example, if one methodis in heavy use, the transmission speed may decrease, and thus thesystem will more likely use the transmission through the other protocol.However, with lower use, the other protocol may actually be theconventionally faster transmission protocol. The system may thereforetake advantage of the faster transmission protocol based on the actualsystem implements and in use at the time of the transmission.

Exemplary embodiments may be used to redirect data traffic to differentnodes within the mesh upon an indication of a failed node. When a datatransmission results in a failure across the redundant transmissionprotocols or interfaces, the system may be configured to send the datapacket through another node. The retransmission through another node maybe though a different communication protocol. In an exemplaryembodiment, the system may select a different communication protocol toaccommodate or correlate to a transmission distance to the new node. Forexample, if the original transmission is over short range, thecommunication protocol for the original transmission may be one forhigher throughput speeds at short range, but which may suffer greatertransmission loss if over longer ranges. If the retransmission toanother node requires longer transmission distances, such as to the nextnode in the chain, the communication protocol permitting for long rangetransmission while maintaining transmission fidelity may be selected forthe retransmission.

Exemplary embodiments described herein may be a consolidatedcommunications device 606 containing a multicore processor, DDR4 RAMmodules, multi gigabit options, and interchangeable solid state harddrives through bus on the motherboard. This processing core can becoupled with additional communication devices such as computing boardswith 802.x, LTE (1.9-3.9 ghz), TV White Space (900 mhz), or WIMAX radiosthrough PCI busses built into the communication device. The architecturein the hardware decided upon gives the system the ability to processtraffic across the networking interfaces in several capacities andconfigurations efficiently through a software based routing systemrunning locally via a virtual machine.

In an exemplary embodiment, the virtual machine may include a modifiedLinux kernel utilizing, for example, Debian core capabilities andlibraries. Linux kernels are the base software nearly all networkingdevices are built upon. Exemplary embodiments described herein mayinclude a Linux kernel configured to create routing tables for eachconnected networking standard that can create individualized routingmarks for bits of data flowing in and out of each communication methodand migrate this data through pre and post routed data tables within thekernel running on virtual machine systems within the core operatingsystem. These bits are may then be processed out of the virtual machine,into the core kernel and in/out of the physical hardware attached to thedevice. This allows the system to bond and aggregate data flowingthrough the compiled system in multiple option sets as chosen by theuser or system configuration.

By utilizing both pre and post routing marks, specifically marked datacan pass across the selected communications standard to the next devicein the chain allowing it to flow to the devices or routed systemsfurther up the network until it is processed or received at its desireddestination. The system can pass these marked data bits individually,collectively, or aggregated all through the same tunnels increasingperformance and reliability of the entire networking stack. This alsoallows for redundancy across a different standard if the data bit doesnot successfully travel down the marked pre and post routed paths.

Mesh networks are technically designed to redirect traffic across thesame networking standard to self-heal a device that is not functioningcorrectly. Unlike these mesh networks, exemplary embodiments describedherein allow the same device to route unsuccessful traffic across adifferent standard to the next device, and then move it to theappropriately marked standard to continue the delivery of the data bit.In the event of total device failure, utilizing standards like Long-TermEvolution (LTE) and TV Whitesapce (TVWS) allows a better delivery forlarger distances to heal the network. Therefore, even if an entire nodefails, the next node or another node may be in range to pick up thetransmission. Even though these standards have slower throughput optionson their respective frequencies, it allows the bit to travel moreefficiently over distance and heal back onto the appropriately markedstandard.

Exemplary embodiments may be used to drastically reduce networkingtraffic bottlenecks, and latency across the entire network may bereduced by allowing the devices processing power to manage these routingtables through multicore processors instead of forcing the entirenetwork to convert to a slower throughput standard until the device isreplaced. Bottlenecks may be contained within one device withsignificant processing power. This especially shows in streaming andbuffering environments, as the processing power of the aggregate of alldevices are allowed to essentially cluster process the network trafficand share hardware resources to improve an efficient pipe for the dataflowing across the network.

In an example environment of a casino, or small city, a system mayinclude 100 communication nodes to define a system deployed utilizing anLTE radio, a 802.11N 2.4 GHz and 5 GHz radio, and an 802.11AC 5 GHzradio setup within each unit. LTE’s distance allows the system to createa packet mark of 1 for backhaul traffic, and 2 for traffic intended toflow specifically across the LTE frequency. The system may be configuredto use a packet mark of 3 for 2.4 GHz traffic, 4 for 5 GHz N traffic and6 for AC traffic. Through the USB bussing built onto the motherboard inthe system, the system may connect a surveillance camera and pass thefeed of the camera across packet mark 2 to be carried through thenetwork and home to the VMS system across the marked 1 tunnel. Each unitmay carry ping traffic to the next unit across packet mark 1 forredundancy triggers within the routing tables. All public traffic(0.0.0.0/0) routed traffic may also be packet remarked 1 upon enteringthe routing table to be carried to the pipe connection to the world ofthe facility. The system may be configured through the routing table toping 5 GHz traffic between each unit on packet mark 4 and tell thesoftware to move traffic marked 2,1 that is undeliverable back acrosspacket mark 4 and out through the marked 1 tunnel. Traffic marked 3 isfor public connectivity (guest WiFi, etc). Should traffic marked 1 beundeliverable it may travel across marks 4 and 6 to the next unit. Thisunit’s routing table will remark this traffic mark 1 and continue todeliver the traffic across the back haul tunnel created until thetraffic reaches the end destination to travel out to the 0.0.0.0/0tunnel in the last/first routed device to the internet.

For aggregation, an exemplary embodiment of the system may bond thesemarks in the routing table to be processed at each device’s virtualmachine (VM) (multicore processor, massive RAM, SSD). M2+M3+M4+M5+M6->M1happens at the processor and passes through the hardware as one packetset to the next device until 0.0.0.0/0 is reached. All of this may beinterchangeable in the routing tables. M1 can be assigned to 802.11ACwhich is the fastest throughput channel available if the devicedistances are close. In a long distance scenario (10-15 miles betweendevices) M1 could be passed across TVWS. However, this would reducethroughput speeds where 0.0.0.0/0 are met. Latency and redundancy may bemaintained through the chain and pipes created in the routing tables. Inredundancy, the table simply shifts. M2+M3+M4-M5+M6->M4+M1 and the nextdevice would process this into the routing table and outputM5+M2+M3+M4+M6->M1 to the next device which would place it back into thepacket mark order that is preset as the base order. In the event of M1undelivered traffic, the output would then become -M1+M2+M3+M4+M5+M6->M4and delivered to the next device that would then place it back into thecorrect sequence.

Exemplary Features

Exemplary embodiments may include a video management system. The systemmay be configured to record one or more tags (that may include anyreceived data, marker, identifier, information, etc.) and associate thetag with the captured data stream from the camera. For example,referring to FIGS. 2 and 7 , the system 200, 700 may include one or moretransmitter/receivers 214, 714 configured to ping a mobile electronicdevice 716 of a user 718 and retrieve an identification number from themobile electronic device of the user, such as a MAC address, or otherdevice ID. The system may be able to detect electronic identifiers fromdifferent electronic devices 716, such as, for example, smart phone716A, smart watch 716B, tablets, e-readers, laptops, pedometers,personal monitors, sensors, ID tags, etc. Exemplary embodiments mayinclude a database management system and/or metadata generation thatstores the tag, such as the device ID, with the data stream from acamera, thereby associating an identification in proximity to a camerawith the camera data stream and/or frames of the camera at the time thetag is detected. Exemplary embodiments may be configured to detect asignal strength of the tag source (such as a signal strength from amobile device during detection of the device ID) such that detection ofthe tag from multiple nodes may be used to triangulate and/orapproximate a location of the device relative to the plurality of nodes.Exemplary embodiments may therefore capture and/or store a tag, such asa device ID (e.g. MAC address), timestamp associated with the detectedtag, signal strength, related node detecting the tag, or closest cameraassociated with the detected tag.

Exemplary embodiments of the video management system may be used toretrieve associated data stream(s) corresponding to a desired tag, e.g.device ID. The system may be able to generate a video stream from aseries of data streams captured from different cameras of the system inwhich a desired tag is detected. The different data streams that have adetected tag associated therewith may be played or combined sequentiallybased on the time a tag, such as the device ID, is detected and/or asignal strength of the device ID relative to the camera and/or nodeassociated with the camera.

Exemplary embodiments may include additional features. For example,during transmission, the system may be configured to detect the presenceof an unauthorized addressed device within the system. For example, if atag is detected associated with an unauthorized user or personnel, theunauthorized tag may trigger the system to send an error notificationand/or stop transmitting data packets. The system may be configured tosend notifications, security alerts, or otherwise identify unauthorizedaccess.

Exemplary embodiments described herein may include any combination offeatures, benefits, and components described herein, including, withoutlimitation (in any combination): reduction of 40%-60% of traditionallabor installation costs, reliable WiFi (or other transmission protocol)connectivity with TVWS (or LTE or other combination of transmissionprotocols) backup, virtual storage software, onboard 1T of storage (orother desired amount) per node, encrypted storage to increase dataprotection, cloud storage backup, Power over Ethernet installation,decentralized network to reduce or eliminate IDF or Server Rooms, and/ora one box turnkey surveillance solution.

Exemplary embodiments of the system described herein can be based insoftware and/or hardware. While some specific embodiments of theinvention have been shown, the invention is not to be limited to theseembodiments. For example, most functions performed by electronichardware components may be duplicated by software emulation. Thus, asoftware program written to accomplish those same functions may emulatethe functionality of the hardware components in input-output circuitry.The invention is to be understood as not limited by the specificembodiments described herein, but only by scope of the appended claims.

Although embodiments of this invention have been fully described withreference to the accompanying drawings, it is to be noted that variouschanges and modifications will become apparent to those skilled in theart. Such changes and modifications are to be understood as beingincluded within the scope of embodiments of this invention as defined bythe appended claims. Specifically, exemplary components are describedherein. Any combination of these components may be used in anycombination. For example, any component, feature, step or part may beintegrated, separated, sub-divided, removed, duplicated, added, or usedin any combination and remain within the scope of the presentdisclosure. Embodiments are exemplary only, and provide an illustrativecombination of features, but are not limited thereto.

When used in this specification and claims, the terms “comprises” and“comprising” and variations thereof mean that the specified features,steps or integers are included. The terms are not to be interpreted toexclude the presence of other features, steps or components.

The features disclosed in the foregoing description, or the followingclaims, or the accompanying drawings, expressed in their specific formsor in terms of a means for performing the disclosed function, or amethod or process for attaining the disclosed result, as appropriate,may, separately, or in any combination of such features, be utilized forrealizing the invention in diverse forms thereof.

What is claimed is:
 1. A distributive memory storage system, comprising:a data generating source configured to generate a data streams; aplurality of nodes in which each of the nodes of the plurality of nodesincludes a processor and memory, and is configured to receive thegenerated data stream.
 2. The distributive memory storage system ofclaim 1, wherein the system is configured to fragment the data stream,and store different fragments on different nodes of the plurality ofnodes.
 3. The distributive memory storage system of claim 2, wherein thesystem is configured to associate metadata to a fragment of the datastream providing instructions on how to recreate the data stream fromthe fragment.
 4. The distributive memory storage system of claim 3,wherein the metadata indicates a node to store the fragment but not aspecific address of a memory location being used for the fragment. 5.The distributed memory storage system of claim 4, wherein a node isconfigured to receive the fragment and store the fragment at the memoryof the node if the metadata indicates the node or pass the fragment toanother node if the metadata indicates the fragment is not for the node.6. The distributed memory storage system of claim 2, wherein the systemis configured to save the fragments in file formats in which a memorylocation corresponding to a fragment is unknown to other nodes of theplurality of nodes except the node in which the fragment is stored. 7.The distributed memory storage system of claim 2, wherein a mapping ofspecific memory locations is managed locally on a local node providingthe memory for a specific fragment of the data stream.
 8. Thedistributive memory storage system of claim 1, wherein the system doesnot include a centralized data storage location.
 9. The distributivememory storage system of claim 1, wherein the plurality of nodes areconfigured to store portions of the data stream to define adecentralized storage network.
 10. The distributive memory storagesystem of claim 1, further comprising a memory controller associatedwith each node of the plurality of nodes, wherein the memory controlleris configured with a naming convention that identifies available memoryfor each of the plurality of nodes without knowing a specific memorylocation within the node.
 11. The distributive memory storage system ofclaim 10, wherein the memory controller is configured to have access tothe memory and specific memory locations available to the nodeassociated with the memory controller.
 12. A method of storing data in adistributive memory storage system, comprising: receiving a data stream;providing a plurality of nodes in which each of the nodes of theplurality of nodes includes a processor and memory, and is configured toreceive data; and storing at least a portion of the data stream at anode of the plurality of nodes.
 13. The method of claim 12, furthercomprising fragmenting the data stream; and storing different fragmentson different nodes of the plurality of nodes.
 14. The method of claim13, further comprising associating metadata to a fragment of the datastream providing instructions on how to recreate the data stream fromthe fragment.
 15. The method of claim 14, wherein the metadata indicatesa node to store the fragment but not a specific address of a memorylocation being used for the fragment.
 16. The method of claim 4, furthercomprising receiving a first fragment of the data stream at a firstnode; determining at the first node whether the metadata of the firstfragment indicates the first node as a storage location of the firstfragment, and storing the fragment at the memory of the node if themetadata indicates the first node or passing the fragment to anothernode if the metadata does not indicate the first node.